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ABSTRACT 



In this paper a new item analysis index, e, is derived as a function 
of difficulty and discrimination to represent Item efficiency. In this paper 
item discrimination is not Independent of Item difficulty end it is demon- 
strated algebraically that the maximum discriminating power of an item may be 
determined from its difficulty. Item efficiency it. defined as the ratio of 
observed discrimination to maximum discrimination. 

Tho e-index will range from zero to unity anti will provide additional 
information for item analyses. Its probability interpretations may provide an 
attractive psychometric criterion for the retention or rejection of items. 
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In a typical analysis of z test item two Indices are usually computed » 
a difficulty Index and a discrimination Index* If one assumes an analysis 
based upon the performance of two groups on the item, typically referred to as 
a tM. analysis, then a two by two contingency table may b& used in the tabula* 
tion of the indices* Such an approach would typify the approaches suggested 

m 

by Kelley ( 1939 ), Johnson (1951) and Cureton ( 1957 ) and is discussed In almost 
a?\y basic measurement text devoting some space - to item analyses. 

Assume that individuals have responded to some item in either a 
positive fashion* r, or a negative fashion; w. Furthermore assume that either 
on the basis of their total scores on the Instrument associated with the item 



or on the basis of some outside criterion two equal groups, and de- 

ter mi nod from NT . In tills 'case & and cj^ in total represent N Individuals 
where N may be equal to or proportionate to N‘ . Then using the subscripts X 
and 2 to denote those symbols associated with gj and c^, respectively, the N_ 
responses to the item arc presented in Table 1. 



Table 1 

Contingency Table Summarization of Two Group Responses 

to a Single Item 



aersLxa? 

Response 


'-ytii sttsssitt nsw-w xtusmesp,! jwasrwuu . m ft, s wmr 

Group 


rraatagaMtaar**. 


Category 


JD 


3 2 


Total 


Positive 


*1 




ri+y z 


Negative 


Wi 


w 2 


Wj+W 2 


Total 


r\W\ 


r 2^2 

mm** «■ 


N 



The difficulty Index,, a, of the Item may be denoted as: 
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and it may range from zero to unity. One interpretation of a. is that it rep- 
resents the proportion of N_ individuals respond! riy positively to the item. A 
second Interpretation is that represents the probability of observing a oosi 
tlve response to the item, P(r). 

The discrimination index of the item, b, may be denoted as: 
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Where 



thu 



ii*% = a hi 
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and it may range from positive to negative unity. One interpretation of b is 
that it represents the difference between two conditional probabilities, the 
probability of a correct response given membership in group one, jKnl&j) > less 
the probability of a correct response given membership in group two, £(r|<k)« 
In a very special sense t.ha marginals of Table 1 are fixed and there 
is an interdependence between difficulty and discrimination, Tn this paper 
difficulty is chosen to be an independent variable while discrimination is 
chosen to be the dependent variable. That Is to say# discrimination Is as- 
sumed to be a function of difficulty and thus Its magnitued for any item is 
tempered by the magnitude of Hie item 1 * difficulty index. 

A frequent problem encountered by "users " of Item analyses is one of 
Interpreting both Indices, difficulty and discrimination, simultaneously and 

V 

making a decision about the disposition of an Item, either retaining or rejact- 
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ing It for future use. All too frequently interpretations are confused when 
either or both Indices departs even slightly, from .BO for difficulty and pos- 
itive or negative unity for discrimination., (It should be noted here that it 
is a popular misconception that the Ideal difficulty index should be .50. For 
a comprehensive discussion of this point see Henrysson, 1971.) 

The major objective of paper Is one of deriving a new index that 
will facilitate the interpretation of Item analyses. In this paper a new in- 
dex, is presented as a function of difficulty and discrimination. Conceiv- 
ably the £ index might facilitate as much if not more Information than the 
simultaneous interpretations of difficulty and discrimination while at the 
same time it should be less confusing as it may be interpreted within a simple 
p robabi 1 i ty framework . 



Maximum Discrimination as. a. Function of Difficulty 

As noted the discrimination index may be thought of as being dependent 
upon the difficulty Index. When discussed genetically the discrimination in- 
dex is Incorrectly assumed to range from positive to negative unity. Regard- 
less of the type of discrimination index used the absolute maximum value of 
unity can be attained only when the associated difficulty Index is .50. 

• As the difficulty index of an Item deviates -.rom .50, either above or 



below It* the maximum ceiling of the discrimination index is reduced from uni- 
ty. For each metric unit of deviation from .50 for a difficulty Index there 
is a two metric unit reduction from unity for the maximum celling of the asso- 
ciated discrimination Index. (See Hofmann, 72# for greater detail on this.) 
Thus given a difficulty index of a. its absolute deviation from .50? jJDj? may 
be used to compute the celling or maximum possible discrimination Index, In 
absolute value terms |bj*| » of the associated discrimination index. 
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and ’ v * 

ifcri - i.oo - [4] 

Logically the principle Involved In computing maximum discrimination 
is presented by equations 3 and 4, however the pragmatics of the concept are 
obscured by the equations. A rnore reasonable set of equations for maximum 
discrimination may be utilized if one is willing to be cognizant of the direc** 
tion of the difficulty deviation. Assume that (a * ,50) then maximum discrim- 
ination may be defined algebraically as; 



b* « Zh. ' (5) 

Assume that (a^ * .50) then maximum discrimination may be defined as: 
b* = 2(1 -a). [6] 



Geometrically maximum discrimination has a perfect curvilinear rela- 
tionship to difficulty within the four quadrants of a two dimensional space. 
Within any one quadrant maximum discrimination Is linearly related to diffi- 
culty. Because of an isomorphism between quadrants the nonadjacent quadrants 
are reflections of each other and any pair of adjacent quadrants may be used 



to depict the relationship between difficulty and maximum discrimination. 

In Figure 1 two adjacent quadrants of a Cartesian coordinate system 
have been depicted. The abclssa of this system represents a difficulty con- 

1 

tlnuum while the ordinate of the system represents a “dual signed* discrimina- 
tion continuum, the values may bo interpreted as either positive or negative. 

The origin is denoted on the difficulty continuum as a .50 so that any 
movement along the continuum will represent directed deviations from .50. The 
dashed line of demarcation within the ’‘left quadrant” represents the lino that 
is defined by any sat of coordinates (a.*b*) whore (a $ .50) and b* Is a maximum 
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discrimination value defined on the discrimination continuum. The dashed 11ns 
in the "right quadrant” is the line that Is defined by any set of coordinates 
(a>k*) where (a. * .50) and b* is a maximum discrimination index, a value de- 
fined on the discrimination continuum. 

" Figure 1. General Cartesian Co-ordinate System 
Defined by Difficulty and Discrimination 
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In Figure ?. the terminus of an item vector, k., has been plotted with 
respect to the difficulty of the item, a, and the discrimination of the item, 
b. A second item vector, k*, has been plotted with respect to the difficulty 
of the item, a, and the maximum discrimination, b* 9 of the Item. 

Assume the Roman letter o. is associated with the origin. There are 
two right triangles depicted in Figure 2, ako and ak*o. Both triangles have 
the same base, the deviation of the difficulty index a, from .50. The triangle 
defined by ak*o t being the ideal triangle, will always be larger than triangle 
ako, the observed triangle. Hie ratio of the areas of the two triangles will 
indicate the size of ako relative to the maximum size it might have obtained. 
That is, the ratio of the two areas may be thought of as representing the ef- 
ficiency of the Item. The better an item functions the more closely will the 
ratio of the two areas approach unity. 

Figure 2. Item Vector k w Defined by Co-ordinates (a,b) and Ideal Item 
Vector ji*’’Defined by Co-ordinates { 4 ^*;. 
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Technically the area of the observed triangle Is a function of the 
types of discriminations the Item makes- In referring to Table 1 It may be’ 
observed that In actuality {j r^+r^) Individuals are judged as being better than 
(wi+ws) other Individuals. However, the item functions as though [(rj+r?) 

(wj+w^)] dichotomous discriminations are made. The absolute maximum number of 
discriminations which may be made for a given item Is, (N / 4 ). 

Consider the component parts of. the equation 

{&+r ■ IiWi + £ili z * I2H1 + I2W2; [ 7 ] 

then It is possible to consider the concept of "proper’* and "improper" dis- 

* 

criminations. Let the term proper discriminations refer to those point dis- 
criminations which are desirable In the sense that they result through a maxi- 
mizing of the frequency of one particular response type, positive response, in 

/ 

one particular group, group one, while the other response type, negative re- , 

sponse, is being maximized in the other group, group two. The term improper 

1 

discrimination may be associated with those point discriminations which are ' 

not desirable in the sense that they occur as a result of undesirable response ^ 

types occurring in both groups. For any item the number of proper discrimina- 
tions is characterized by (r^) and the improper discriminations by ( r ? Wi ) . 

Implicity for easy items the negative responses should all be accrued by group 
two, W2, and for difficult items the positive responses should all be accrued 
by group one, rj. 

A discrimination index in terns of point discriminations, {b}, Is just 
the difference between proper and improper discriminations. 

{ b } * (raw*.) - ( r 9 Wi ) [81 

end the relative discrimination index is given by 
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The maximum discrimination Index, however, assume no Improper dis- 
cn ml nations. Either r* or wi is assumed to bo zero and either £1 or w* Is 
assumed to be ru Thus, maximum discrimination in terms of point discrimina- 
tions* {ib*}, represents the maximum number of proper discriminations possible 
for a qiven N and difficulty index. 

{b*} * (rj+rj)^; a, < .50 D0-?3 



or 

{b? - (wi*Wp>)n; a > .50 

and the relative maximum discrimination index is given by 

{b*} 

~W~ 



[10-b] 



b* * 



[ 11 ] 



Now given that the area of any triangle is equal to one-half the base 
multiplied by the altitude and given that both triangles in Figure 2 have the 
same base then the ratio of their areas is algebraically equivalent to the 
ratio of their altitudes. Alternatively the ratio represents the number of 
observed proper discriminations less observed improper discriminations divided 
by the maximtfin possible number of proper discriminations. The ratio of {b.} to 
{ib*} will range from zero to unity, assuming Tb> Is positives and may he thought 
of* conceptually, as representing the “purity” of the dl$ criminations made or 
the efficiency of the item. Let e represent a general efficiency Index then: 
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FT 

and In modified form: 
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When one considers the fact that (b*3 will always be positive and {b} 
may be positive or negative then it becomes Immediately apparent that £ as de- 
fined by equations 12 and 13 may be positive or negative. When e_ is negative 
it is negative because more improper than proper discriminations were made. 

The terms proper and improper were somewhat arbitrarily assigned to two quan- 
tities on the assumption that more positive responses and, hence, fewer nega- 
tive responses would always be made by group one relative to group two. When 
this assumption is not met the e index will be negative; However, for inter- 
pretations within the framework of proportions and areas the sign of may be 
neglected. The negative sign of e becomes meaningful only within the frame- 
work of probability. 

Given the conditional magnitude of the difficulty index the general e 
my be further specified as: 

b 

Oi R “gr— 5 2,$ *50 • [14] 

general efficiency for Items having difficulty Indices less than or equal to 
.50 and: 

# # 

to" *TOT *■ ** - 50 t163 

general efficiency for Items having difficulty Indices greater than or equal 
to . 50 . 

Certain initial observations may be made with respect to e and propor- 
tion Interpretations 

(a) If the observed discrimination index of an item is zero then the ef- 
ficiency of the item is zero. 
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(b) for any level- of difficulty* excluding zero and unity* It Is theoreti- 
cally possible for e to range from zero to unity assuming a positive 
discrimination Index. 

(c) Efficiency Is the ratio of observed proper discriminations less 1a- 

* proper discriminations to the maximun possible nufiwer of proper dis- 
criminations for a given difficulty level and group size. 

(d) The general index e. is indicative of how well an item has functioned 
relative to how well it might have functioned for a given N and spe- 
cific difficulty level. 



Probability Interpretations of Efficiency 

In the previous section it was noted that the general efficiency index 
could be subdivided into two indices, one for items having difficulties less 
than or equal to .50, henceforth efficiency of the first kind, ej, and one for 
items having difficulties greater than or equal to .50, henceforth efficiency 
of the second kind* e*. The indices of efficiency may be further utilized to 
make probability interpretations with respect to positive responses and with 
respect to negative responses. 

• Equation 14 defining ej* for Items having difficulties less than or 
equal to .50* may be modified to define a computational equation for re- 
gardless of item difficulty and sign of the discrimination index. 



' ra-r 2 

^ ri+ra> 

*» .i * jpwi 



[16] 



Similarly equation 15 defining e*, for Items having difficulties greater than 
or equal to .50, may be modified to define a computational equation for re 
gardless of Item difficulty »nd sign of the discrimination Index. 
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Utilizing aquations 16 and 17 It is possible to discuss conditional 
probabilities and note that quite unlike traditional U-L discrimination in- 
dices efficiency considers two events which are mutually exclusive and ex- 
haustive with respect to a given sample space, 

Assume that a positive response has been made to an Item. Given an 
individual making a positive response the probability that the individual Is a 
member of jgj is given by £.{.SLijr) while the probability that the individual is 
a member of & is glven’by P^jr), where: 



efficiency of the first kind Is the difference between two conditional proba- 
bilities, where the probabilities are for group membership, either & or 
given a positive response. 

Assume that a negative response has been made to an item. Given an 
Individual making a negative response to an Item the probability that the in- 
dividual Is a member of & Is given by P(£i|w) while the probability that the 
individual Is a member of & Is given by P(fl 2 |w), where: 




[ 18 ] 



and 



/ 




[19] 



Then 



Si ■ £.(5ilr) - B&tlr) 



[ 20 ] 







[21] 
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Then: 




im 



£2 “ £{32 |w) •' P-Cjaa 1 st) 




efficiency of the second kind Is the difference between two conditional proba- 
bilities, the probability of group membership given a negative response* 
Efficiency of the first kind and efficiency of the second kind are 
both mutually exclusive and exhaustive with respect to sample space: 



e.1 * p(a> !r) - £(&ilr)5 


[24] 


1.0 » Pfai |r) + P(g 2 |r); 


[25] 


& ° p(a>Jw) - p.(a.i 1 w) j 


[26] 


1-0 T iifaalw) + £(Sj|w)- 


[27] 


Following logically from these equations are: 




. Si^l 

£.<ai!i) ° ■~t'— 5 " 


[28] 


£(32 Ir) * — i — i 


[29] 


1 -S? 

£(ailii) 3 ~T~~> 


[30] 


02+1 

£(52 Is) 3 -£— • 


[31] 



Such additional probability interpretations for efficiency should bo 
quite appealing for certain In depth analyses of items* In Table 2 values for 
equations 20-31 are given for 100 efficiency Indices between zero 'and unity. 

For a negative e column headings may be interchanged so that the first 
column becomes the probability of the second event and the second column be- 
comes the probability of the first even. Similarly the odds columns should be 
Interchanged* 
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Table 2 

Probability Interpretations Associated with 100 

Efficiency Indices between Zero and Unity 



e 


A AJaCtTBS^ X-JP%uS»" 1 flf C 8 SP 

Probability of Probability of 

K1 rst Event Second Event 


mi mm tunwifiMwi 

Odds for Odds against 

First Event First Event 


.00 


.500 


.500 


1.000 


1 .000 


.01 


.505 


.496 


1.020 


.980 


.02 


.510 


.490 


1.040 


.960 


.03 


.515 


.485 


1 .061 


.941 


.04 


.520 


.480 


1.083 


.923 


.05 


.525 


.475 


1.105 


,904 


.06 


.530 


.470 


1.127 


.886 


.07 


.535 


.465 


1.150 


.869 


CO 

o 

• 


.540 


.460 


1 'o 


.851 


.09 


.545 


.455 


1.197 


.834 * 


.10 


.550 


.450 


1.222 


.818 


.11 


‘ .555 


.445 


1.247 


.801 


.12 


.560 


.440 


1.272 


.785 


.13 


.565 


.435 


1.298 


.760 


.14 


.570 


.430 


1.325 


.754 


.15 


.575 


.425 


1 .352 


.739 


.16 


.580 


.420 


1.280 


, 724 


.17 


.585 


.415 


1.409 


.709 


.18 


<590 


.410 


1.439 


.694 


.19 


.595 


.405 


1.469 


.680 


.20 


.600 


.400 


1.500 


•666 


.21 


.605 


.395 


1.531 


.652 


.22 


.610 


.390 


.1.564 


.634 


.23 


.615 


.385 


1.597 


.626 


.24 


.620 


‘.380 


1.631 


.612 


.25 


.625 


.375 


1.666 


•600 


.26 


.630 


.370 


1.702 


.587 


.27 


.635 


.365 


1.712 


.574 
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Table 2 (cent.) 



$ 


Probability of 
First Event 


Probability of 
Second Event 


Odds for 
First Event 


rr imimi u rimriir 

Odds against 
First Event 


,2$ 


.640 


.360 


1.777 


.562 


.29 


.645 


.355 


1.816 


.550 


.30 


.650 


.350 


1.857 


.538 


.31 


.655 


.345 


1.898 


.526 


.32 


.660 


.340 


1 .941 


.515 


.33 


.666 


.335 j 


1.985 


.503 


.34 


.670 


.330 


2.030 


.492 


.35 


.675 


.325 


2.076 


.481 


.35 


.680 


.320 


2.125 


.470 


.37 


.635 1 


.315 


2.124 


.459 


.33 


.690 


.310 


2.225 


.449 


.39 


.695 


.305 


2.278 


.438 


.40 


.700 


.300 


2.333 


.428 


.41 


.705 


.295 


2.389 


.418 


.42 


.710 


.290 


2.448 


,408 


,43 


.715 


.285 


2.508 


.401 


.44 


.720 


.280 


2.571 


.398 


.45 


.725 


.275 


2.636 


.368 


.46 


.730 


,270 


2.703 


.369 


.47 


.735 


.265 


2.773 


.360 


.48 


.740 


,260 


2.846 


.351 


*49 


.746 


.255 


2.921 


.342 


.50 


.750 


.250 


3.000 


.333 


*51 


.755 


.245 


3.081 


,324 


.52 


.760 


.240 


3.166 


.315 


.53 


.765 


.235 


3.255 


.307 


*54 


.770 


.230 


3.347 


.298 


.55 


.775 


.225 


3.444 


.290 


*56 


.780 


.220 


3. 545 


.282 


! « 


.785 


.215 


j 3. 651 


.273 
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Table 2 ( cont .) 



e 


Probability of 
First Event 


Probability of 

Second Event 

.... - -- 


Odds for 
First Event 


Odds against 
First Event 


00 

tr > 

« 


* 

.790 


.210 


| 3.761 


.255 


,59 


.795 


.205 


3,878 


.257 


,60 


.800 


.200 


4.000 


.250 


.61 


.805 


.195 


4.128 


.242 


. 6 ? 


.810 


,190 


4.263 


,234 


.63 


.815 


.185 


4.405 


.226 


,64 


,820 


.180 


4.555 


.219 


.65 


.825 


.175 


4.714 


.212 


.66 


.830 


.170 


4.882 


.204 


.67 


.835 


.165 


5.060 


.197 


.68 


.840 


.160 

• 


5.250 


.190 


.69 


.845 


.155 


5.451 


.183 


.70 


.850 


.150 


5.666 


.176 


.71 


.855 


. 145 


5.896 


.163 


.72 


.860 


.140 


5.142 


.162 


.73 


.865 


.135 


6.407 


,156 


.74 


.870 


.130 


6.692 


.149 


.75 


.875 


.125 


7.000 


.142 


.76 


.880 


.120 


7.333 


.136 


.77 


.885 


.115 


7.695 


.123 


,78 


.890 


.110 


8.990 


.123 


,79 


. 89 $ 


.105 


8.523 


.117 


.80 


.900 


.100 


9,000 


.111 


.81 


.905 


.095 


9.526 


.104 


.82 


,910 


.090 


10.111 


.098 


.83 


. 91 $ 


.,085 


10,764 


.092 


.84 


.920 


.080 


11.500 


.086 


.85 


.925 


.075 


12.333 


,081 


.86 


.930 


.070 


13.285 


.075 


.87 


.935 


.065 


14 . 3 &/ 


.069 
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Table 2 (cont,) 



trav-itoasp 

ft 


Probabi lity of 
First Event 


Probability of 
Second Event 


xmMMOMgttsi uf 11 t *1 <ai i*' '»t*. 

Odds for 
First Evfcrt 


rvtae. cactrw. 

Odds against 
First Event 


.88 


.940 


.060 


15.666 


.063 


.89 


.945 


.055 


17.181 


.050 


,90 


.950 


.050 


19.000 


•052 


,91 


.955 


.045 


21.222 


.047 


.92 


.960 


.040 


24.000 


.041 


.93 


.965 


.035 


27.571 


.036 


.94 


.970 


.030 


32.333 


.030 


.95 


.975 


.025 

« 


39=000 


.025 


*96 


.980 


.050 


49.000 


.020 


.97 


.985 


.015 


65.666 


.015 


.98 


.990 


.010 


99.000 


,010 


.99 


.995 


.005 


199.000 


.005 


1.00 


1 .000 


,000 


mm imam 

— rr r -- - 





It may be noted that and §2 are proportional to each other. The 



ratio, of to Gj represents the odds In favor of a negative response while 
the ratio of £1 to ©2 represents the odds In favor of a positive response. 
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nr™ 


El + l2 


®2 


IU+I2 







[32~a3 



[32~b] 



Such equations are referred to as M odd‘s ratios.” The odds for and against, a 
response type given a particular Index are also reported for the 100 different 
e indices In Table 2. 

Equations algebraically equivalent to equations 32- a and 32-b may be 
derived using the difficulty Index. 
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[33~a1 



$2 * 1 



[33-b] 



If one wishes to convert e* to e 2 ^ only necessary to multiply 
by the odds in favor of a positive response, equation 33-a. The converse is 

true for converting e^ to 



The, Probability of Obtaining an Observed , eby Chance 

The general £ Index has been discussed within the framework of £.1 and 
ej., it was noted that for any given level of difficulty e may range from zero 
to unity. Quite logically one would like to know the probability of obtaining 
an observed e for any given index of difficulty. Generieally, what is a sig- 
nificant e? 

The model contingency table from which a is computed is unique within 
the framework of statistics. Theoretically e is a measure of departure from 
independence in the contingency table. However there is a different probabil- 
ity distribution for e associated with each uniquely different sample size, N» 
and each uniquely different difficulty level, a. In order to compute the 
probabilities associated with any given e it must be assumed that all four 
marginals of the contingency table are fixed. That is to say, the probabili- 
ties reported for any e are determined from the specific probability dtstribu- 

tlon of o associated with a given N, and a- 

Technically In order to test the null hypothesis of Independence* 

(Hq!® * 0)* It Is necessary to compute the probability of obtaining tne ob- 
served e and all possible e Indices of a larger magnitude assuming constant 
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marginals. Although Pearson’s (W2) Chi square test might be used with this 
model of a contingency table it was not designed specifically for such use and 
in using it one would have to constantly keep in mind the consequences of its 
use with small sample sixes and also meet the assumptions of expected frequen- 
cies greater than five in the cells of the table* 

A test designed specifically for the type of model contingency table 
associated with a is Fisher’s (1935) exact test. Essentially Fisher’s test 
would Indicate the exact probability of obtaining an* observed e given a par- 
ticular N and a. Furthermore it could be* used to compute the exact probabili- 
* ties of each associated e^ greater than the observed js. In summing up all of 
these exact* probabilities one would have the probability of obtaining an £ as 
large as or larger than the one observed with the given level of difficulty* a_* 

and group size, N. / 

For any test bavin* more than four* or five items or fifteen or sixteen 
individuals such an approach would be computationally time consuming. H 

However, it is possible to use a variation of Fisher’s test, which is ^ 

based upon the hypergeometrlc distribution, to establish the magnitudes of the 
e Indices which would represent the extreme tails of the distributions of such 
indices for various difficulty levels and group sizes. Thus, it Is. possible, 
so to speak, to establish ’’tables of significance” for the £ index by comput- 
ing the probabilities associated with the most extreme values for e and work, 
computationally, toward the less extreme values. 

. The method suggested here Is an approximation method whose numerical 
methods are reported by Hofmann (72). Essentially this variation involves the 
computation of the probability of the least Independent contingency table, e * 

1.0, for a given difficulty level and then continues to compute the prcbablli- 
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ties for successively more Independent tables, while the difficulty Index Is 
held constant. Such an approach avoids the extremely time consuming computa* 
tion associated with large factorials » which will occur in the more dense, 
middle, area of a symmetric distribution. Further discussion of this proced- 
ure Is beyond the scope of this paper. Using this procedure levels of signif- 
icance for e were computed for difficulty indices between zero and unity for 
samples ranging in size from 8 to 100 for the .05 level of significance ano 
are reported in Table 3. 

The interpretations associated with Table 3 are the same as those made 
for any percentile point, a one-tail test. For a given group, N ? with a given 
difficulty, a, there is a symmetric distribution of possible a indices. The 
tabled e index is that e associated with the percentile point which best ap- 
proximates the .05 percentile point. 
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Table 3 (confc.) 
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Total; Jest Efficiency . 

fhfs paper has been concerned primarily with the computation of thee- 
Index at an Item level. This does not, however, preclude its use as a total 
tost statistic. Oust as one can talk of total test difficulty so also can one 
talk of total test efficiency* In this section the equations for computing 
total test efficiency will be discussed* No attempt will be made to interpret 
the total test efficiency index other than the cursory definition that follows 
from it computationally. 

Assume some test composed of jl i terns. Then Nj individuals will respond 
to item j. The number of individuals in either. & or <£> will be denoted as nj 
for the item and (2nj = if*). The difficulty and discrimination Indices 

for the jth item roay be denoted as and dj respectively. The total number 
of possible discriminations that may be made by the jth item is ^3 . The a&- 



solute maximum frequency of proper discriminations that is possible for the 
jth item, bj , is given as: 
a Njij; 

■ 2 2j«j- C 34 3 

The frequency of proper discriminations less improper discriminations , 
(b} f for the jth item is given as: 





Inasmuch as efficiency is defined, at an item level, as the ratio of 
observed proper discriminations less Improper discriminations to the maximum 
possible number of proper discriminations, for a given difficulty Index and 
group si :e, assume a similar definition for total test efficiency. Let total 
test efficiency be represented by the ratio of total observed proper discrimi- 
nations less Improper discriminations s {B>, to the maximum possible numbar of 



/ 



V 
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proper discriminations for the total test* {§_*). Let £ represent total test 
efficiency* assuming that the difficulty of all Items is less than .50 

{B> 



Si 



[36] 



(B*> 



where 



i ” 1 Ml 
1*1 



[37] 



and 



{b*> = 2 j S3I4; 
1*1 



[38] 



thereby allowing equation 36 to be rewritten in computational form as, 



Ei * 



j Mi 

■M 



z 1 mi 

^ ***j ' ■ * 



[39] 



Such an Index Is indicative of the proportion of ’’quality” discrlmina* 
tions made by a total test given that all items have a difficulty less than 
.50. For items having a difficulty index greater than .50 equation 38 Is 
modified to do Fine 



{£*} « 2 j[ nj(l-aj) • 

1*1 

and equation 39 Is modified to determine as 

ijMi 



[40] 



Ej * 



[ 41 ] 



2 i Sl(1'£|) 

1*1 
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If the £ items form a mastery test then either equation 39 or 41 could 
be used to compute total test efficiency. If the £ items form an achievement 
test assume that s. of the items have difficulty indices less than .50 and as- 
sume that k. of the Items have difficulty Indices greater than or equal to .50, 
then if the items are grouped dichotomous ly according to those 1 tiros having 
difficulty indices less than .50 and those equal to or greater than .50 the 
general efficiency index, E, of the achievement test is given by: 



Experience with this index is stll’l growing and therefore its applica- 
tion to total tests is only of theoretical interest at this time. However, 
the systematic study of E, JE* and £2 might be informative. To wit, either 
or E^ might serve as indicators of discriminatory of difficulty homogeneity or 



general index, £, might serve as some index of internal consistency for one 
achievement test. 



This section is written explicitly for my colleagues who say that they 
are nonstatisticians and want something that will immediately benefit them in 
the Interpretation of their item analyses. 

Allowing a little freedom with subjectivity it is possible to set up 
criteria for determining easy, moderate and hard items as well as nonefficient, 
efficient and ideally efficient items. Let the following inequalities, based 
upon effi cisncy, e. a arid difficulty, a, serve as operational definitions of the 
above terms. 




[42] 



perhaps as some sort of index of Internal consistency for a mastery test. The 



V 



A Pragmatic Sc heme for the Use of e 

mm* Mil 1- — MV T - ttrnmmmm m m ^ »%•■*»*••* mmmmt 



ERiC 



33 



32 



O.QO < e < 0. 50 noneffi dent item 
< 0.50 <L £ 1 0.80 efficient Item 

0.80 < e < 1.00 ideally efficient Item 
0.75 <. 1,00 easy Item 

0* 25 <. £ ± 0 . 75 moderate i tern 
0.00 < a <0.25 hard item 

Because exact intervals have not been made these definitions are* the- 
oretical ly> not mutual ly exclusive., but for practical purposes they may be 
thought of as mutually exclusive and exhaustive with respect to difficulty and 
efficiency. From these definitions It is possible to categorize all items of 
an instrument according to Table 4. 



Table 4 

Efficiency by Difficulty Contingency Categorization 
of Subjective Item Types 



Difficulty 


Type 

Criterion 


Efficiency Levels 








Non-Efficient 


Efficient 


ideally 

Efficient 


Easy 


Efficiency 


0.00 < e < 0.50 

0.75 i a 1 1.00 

------ — 

^ 


0.60 < e < 0.80 
0.75 H1 1.00 


0.80 < a < 1.00 
0.75 1.00 


Difficulty 


Moderate 


Efficiency 


0.00 < e < 0.50 

0.25 7 a < 0.75 
-•*» 


0,50 < a < 0.80 
0.25 7 a '< 0.35 


0.80 < e < 1.00 
0.25 7 a 7 0.75 


Difficulty I 


Hard 


Efficiency 


0.00 < e .< 0.50 
0.00 < 3* < 0.25 

>v>» 


0.50 < e < 0.80 

0.00 7 a 7 0.25 
_ 


0.80 < e < 1 .00 
0.00 7 a "0,25 


Difficulty 



Based upon Table 4 it Is possible to construct within the framework of 
Cartesian coordinates a chart for determining Item quality. In Figure 3 such 
a chart has been constructed. Givsn the difficulty and dlscmfilnatlon of an 
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Figure 3 

Iconic Representation of Contingency Categorization of Subjective Item Types 

±1.0(H a-1.00 
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Item as coordinates it is possible to locate the associate item point. If 
the point falls within vhe working area then the item is a working item. Ad- 
ditionally It is possible to determine the relative difficulty of the working 
item. Although this chart was constructed as a function of the t^-lndex, its 
use actually precludes the computation of such an index. 

Assuming Figure 3 to be a triangular probability distribution it is 
possible to briefly discuss the chance probability of obtaining different item 
types. Note in Figure 3 that there are nine different Item types. The proba- 
bility of any item type’s occurring by chance may be determined as the ratio 
of the surface area associated with each item type to the total surface area 
of the probability distribution. The probabilities for the nine subjective 
item types are reported in Table 5. 



Table 5 

Chance Probabilities of Subjective Item Types 



r\gri.y .*-r.Trr~ys- oasgrg* 

Di fficulty 
Level s 


Efficiency Levels 




Non-Efficient 


Efficient 


Ideally Efficient 


Total 


Easy 


.0625 


.0378 


.0250 


.1250 


Moderate 


.3750 


,2250 


,1500 


.7500 


Hard 


.0625 


.0375 


.0250 


.1250 


Total 


.500 


.300 


.200 


1 .0000 



As previously noted the concept of item typos Is subjective as are the 
arbitrary numerical criteria for defining them. It H possible to construct a 
more detailed chart than Figure 3, as well as a more detailed table of c»a?ica 
probabilities to go with It, However, the objective of this small secticr was 
one of establishing a pragmatic schema for the use of e, which was dons. 
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Conclusion and Summary 

The conclusion tc this paper will provide an efficiency analysis of an 
Item given a difficulty Index of .30, & discrimination index of ,29 and a 
sample size of 14. 

(a) By equations 5 and 13 the general efficiency index of the items is ap« 
pro xi mate ly .42. 

(b) Given from above that (e, * ,42). then the probability of an individual 
being a member of group one given that the individual made a positive 
response is (.71), from either equation 28 or Table 2. 

(c) The odds are 2.45 to 1.0 that a person responding positively to the 
itenrwlll be a member of group one* from Table 2, 

(d) The value of e? may be computed as .18 utilizing equation 32-a. 

(e) Given that (e^ * .18) then the probability of an individual being a 
member of group f.wo given that the individual made « negative response 
Is .89, from either equation 31 or Table 2. 

(f) The odds are 1.439 to 1.0 that a person responding negatively to the 
item will be a member of group two* from Table 2. 

(g) According to Table 2 the observed 8 is not significant.at the .05 level. 

(h) According to Table 4 or Figure 3 this item is a "moderate, non-effi- 
cient item," 

(i) From Table 5 It may be inferred that the probability of getting a mod- 
erate non-efficient item, by chance, is .3/5, more probable than any 
other type. 

The nine interpretations given above are with respect to a single item. 
Had wore Items been Included in the analysis It would have been possible to 
make comparative statements as well as relative judgements of the Items, 
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This paper presents tha basic aspects of the efficiency Index. Whether 
or not the Index will replace or supplement the traditional discrimination in- 



dex remains to be seen. However* It would seem that the general efficiency 



Index has many more statistically compelling properties 



than the traditional 



discri ml nati on i ndex . 
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